Apparatus and method for visualizing loitering objects

ABSTRACT

A method for visualizing loitering objects includes: detecting at least one object determined to have been in a selected area of an input image for a preset time period; obtaining representative still images of each of the detected at least one object in respective time periods during the preset time period; and displaying the representative still images in a time order, or generating a video summary in which images of each of the detected at least one object, respectively included in the representative still images, are displayed together on a single image with indication of the time order.

CROSS-REFERENCE TO THE RELATED APPLICATION

This application claims priority from Korean Patent Application No.10-2014-0146437, filed on Oct. 27, 2014, in the Korean IntellectualProperty Office, the disclosure of which is incorporated herein in itsentirety by reference.

BACKGROUND

1. Field

Apparatuses and methods consistent with exemplary embodiments relate todetection of loitering objects.

2. Description of the Related Art

A method of detecting loitering objects may be used to monitor inadvance persons that plan to invade a particular area to prevent theoccurrence of crimes that may be committed by the persons. However,since results of detecting loitering objects at particular periods oftime are displayed on respective still screens, a relationship betweenthe detected loitering objects may not be easily understood.

SUMMARY

One or more exemplary embodiments include a method of addressing such aproblem that it may be difficult to utilize or analyze collected datawhen it is not figured out whether detected loitering objects are thesame object.

Various aspects will be set forth in part in the description whichfollows and, in part, will be apparent from the description, or may belearned by practice of the presented embodiments.

According to one or more exemplary embodiments, there is provided aloitering visualization method which may include: detecting at least oneobject determined to have been in a selected area of an input image fora preset time period; obtaining representative still images of each ofthe detected at least one object in respective time periods during thepreset time period; and performing at least one of displaying therepresentative still images in a time order and generating a videosummary in which images of each of the detected at least one object,respectively included in the representative still images, are displayedtogether on a single image. The method may further include assigning anidentifier to each of the detected at least one object, wherein theidentifier is displayed along with each of the detected at least oneobject on the representative still images or on the single image of thevideo summary. The method may also further include tracking a path ofeach of the detected at least one object to which the identifier isassigned,

In the generating the video summary, the images of each of the detectedat least one object respectively included in the representative stillimages may be displayed on the single image with indication of the timeorder.

Each of the representative still images may display at least one of afront view, a side view, and a rear view of each of the at least oneobject.

The detected at least one object may be at least one object which hasappeared more than a predetermined number of times in the selected areafor the preset time period.

BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other aspects will become apparent and more readilyappreciated from the following description of the exemplary embodiments,taken in conjunction with the accompanying drawings, in which:

FIG. 1 illustrates a structural configuration of a loiteringvisualization apparatus, according to an exemplary embodiment;

FIG. 2 illustrates detecting a particular moving object according totime, which is performed by the loitering visualization apparatus,according to an exemplary embodiment;

FIG. 3 illustrates displaying loitering of at least one moving object inthe form of a video summary by using representative still imagesextracted in respective time periods, which is performed by theloitering visualization apparatus, according to an exemplary embodiment;

FIGS. 4 and 5 illustrate video summaries and displaying a loitering pathof at least one moving object, which is performed by the loiteringvisualization apparatus, according to an exemplary embodiment; and

FIG. 6 is a flowchart of a loitering visualization method, which isperformed by the loitering visualization apparatus, according to anexemplary embodiment.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS

Reference will now be made in detail to exemplary embodiments which areillustrated in reference to the accompanying drawings, wherein likereference numerals refer to like elements throughout. In this regard,the present embodiments may have different forms and should not beconstrued as being limited to the descriptions set forth herein.Accordingly, the exemplary embodiments are merely described below, byreferring to the drawings, to explain various aspects of the inventiveconcept. As used herein, the term “and/or” includes any and allcombinations of one or more of the associated listed items. Expressionssuch as “at least one of”, when preceding a list of elements, modify theentire list of elements and do not modify the individual elements of thelist.

The following description and accompanying drawings are provided forbetter understanding of the inventive concept. In the followingdescription, well-known functions or constructions are not described indetail if it is determined that they would obscure the inventive conceptdue to unnecessary detail.

The following description and drawings are not intended to restrict thescope of the inventive concept, and the scope of the inventive conceptshould be defined by the appended claims. The terms used in thefollowing description are merely used to describe particular embodimentsof the inventive concept and are not intended to limit the inventiveconcept.

FIG. 1 illustrates a structural configuration of a loiteringvisualization apparatus 100, according to an exemplary embodiment. FIGS.2 through 6 illustrate a function of each element of the loiteringvisualization apparatus 100.

The loitering visualization apparatus 100 includes an object detector110, an object identifier 120, a path tracker 130, a representativeimage obtainer 140, and a loitering visualizer 150. Additionally, theloitering visualization apparatus 100 may further include a videosummarizer (not illustrated).

The object detector 110 detects at least one moving object that appearscontinuously or stays for a preset period of time or longer in aparticular area of an input image, by using an object loiteringdetection method. Here, a certain object appears more than apredetermined number of times for the preset period of time may bedetected as the at least one moving object that appears continuously orstays for the preset period of time or longer. Also, the input image maybe a collection of still images or a moving image. The object loiteringdetection method is a method of sequentially detecting a path in whichan object loiters in a particular area. A loitering trace may bedisplayed on a single screen, that is, a single image frame.

FIG. 2 illustrates detecting a particular moving object from an inputimage 200 according to time, which is performed by the object detector110, according to an exemplary embodiment It is identified that theparticular moving object at a point A in S210 is extracted from theinput image 200 at a time of t1, the particular object at a point B inS220 is extracted from the input image 200 at a time of t2, and theparticular moving object at a point C in S230 is extracted from theinput image 200 at a time of t3.

The object identifier 120 assigns an identifier to a detected movingobject. For example, referring to FIG. 2, when the object detector 110first detects the moving object at the point A in S210 at the time t1,the object detector 110 may assign an identifier ID#1 to the detectedmoving object.

The path tracker 130 is configured to track a path of at least onemoving object with an identifier. Referring to FIG. 2, the path tracker130 tracks a path via which the detected moving object has passedthrough the point A in S210 at the time t1, the point B in S220 at thetime t2, and the point C at the time t3 in S230, by tracking thedetected moving object with the identifier ID#1.

The representative image obtainer 140 obtains a representative stillimage of the detected moving object in each time period, while the pathtracker 130 tracks a path of the detected moving object with theidentifier. As an example, representative still images respectively in atime period from 9:00:00 to 9:05:00, in a time period from 9:05:01 to9:10:00, and in a time period from 9:10:01 to 9:15:00 are obtained. Inthis case, the representative still images of the detected moving objectmay be in the form of at least one of a front view, a side view and arear view of the detected moving object.

Referring to FIGS. 2 and 4, a first still image obtained when the movingobject is detected at the point A in S210 at the time t1 of 09:00:28 maybe set as a representative still image in the time period from 9:00:00to 9:05:00, a second still image obtained when the moving object isdetected at the point B in S220 at the time t2 of 09:00:53 may be set asa representative still image in the time period from 9:05:00 to 9:10:00,and a third still image obtained when the moving object is detected atthe point C in S230 at the time t3 of 09:01:07 may be set as arepresentative still image in the time period from 9:10:01 to 9:15:00.

The loitering visualizer 150 collects the representative still imageseach obtained in each time period with respect to each moving objectwith the identifier and displays the representative still images in anorder of time. Referring to FIG. 2, the loitering visualizer 150displays the first to third still images as obtained in theabove-described manner, in an order of time.

Referring to FIGS. 4 and 5, the loitering visualizer 150 collects anddisplays in an order of time representative still images each obtainedin each time period with respect to each moving object with anidentifier. For example, the loitering visualizer 150 may displayrepresentative still images of a moving object 410 or 510 to which anidentifier ID#1 is assigned and representative still images of a movingobject 520 to which an identifier ID#2 is assigned, in an order of time.The loitering visualizer 150 may provide identification information,which may identify an object, to be displayed in each representativestill image obtained in each time period.

The loitering visualization apparatus 100 may further include the videosummarizer (not illustrated) for generating a video summary based on thecollected representative still images obtained in respective timeperiods with respect to at least one moving object that is assigned anidentifier. The video summarizer may be included in the loiteringvisualizer 150. FIG. 3 illustrates display of loitering of at least onemoving object in the form of a video summary by using representativestill images obtained in the respective time periods, which is performedby the loitering visualization apparatus 100, according to an exemplaryembodiment. Referring to FIGS. 2 and 3, the video summarizer displaysimages of the detected at least one moving object respectively includedin the representative still images together on a single screen, that is,a single image or single image frame. Various methods may be employed togenerate the video summary. According to an exemplary embodiment, thevideo summarizer may select one of the three representative still imagesshown in FIG. 2, extracts only a portion of the detected moving objectfrom each of the other two representative still images, and merges theextracted portions of the detected moving object with the selected oneof the three representative still images.

FIG. 4 illustrates that the video summarizer displays images of thedetected moving object 410, respectively included in the representativestill images, together on a single screen 420, that is, a single imageor single image frame, and FIG. 5 illustrates that the video summarizerdisplays images of two moving objects 510, 520, respectively included inthe representative still images, together on a single screen 530. Here,the single screen 530 may be an image generated by combining sixdifferent still images. As described above in reference to FIGS. 2 and3, in order to display six different still images of two moving objects510, 520 captured at six different time points on the single screen 530as a single image frame, various methods may be employed. According toan exemplary embodiment, the video summarizer selects one of the sixstill images, extracts only a portion of the moving object 510 or 520from each of the other five still images, and merges the extractedportions of the moving object 510 or 520 with the selected one of thesix still images.

Additionally, the video summarizer may be configured to further displaya moving trajectory in S410 shown in FIG. 4 and moving trajectories inS510 and S520 shown in FIG. 5 for showing a path of a moving object byusing representative still images obtained in respective time periodswith respect to each object with an identifier. In this case, the videosummarizer may be configured to display a path of a moving object in anorder of time. Alternatively, the video summarizer may be configured todisplay time in S521, S522, and S523, at which a representative stillimage is extracted, on the representative still image of the movingobject in each time period.

Referring to FIG. 5, an example of displaying a time of 09:00:01 inS521, a time of 09:00:32 in S522, and a time of 09:01:03 in S523, atwhich representative still images are extracted, on the representativestill images of the moving object 520, to which the identifier ID#2 isassigned is shown, respectively.

FIG. 6 is a flowchart of a loitering visualization method, which isperformed by a loitering visualization apparatus, according to anexemplary embodiment. The loitering visualization apparatus may beimplemented in a digital video recorder (DVR), a digital video disc(DVD), a network video recorder (NVR), a smartphone, a thin terminal, ahand-held terminal, an image processing apparatus that includes amonitoring camera, or an image playback apparatus.

In operation S610, the loitering visualization apparatus detects atleast one moving object that appears continuously or stays in aparticular area of a received input image or an input image that isbeing photographed for a preset period of time.

According to an exemplary embodiment, if at least one moving object thathas been in a particular area for a time period is detected, thedetected at least one moving object may be further displayed in the formof a rectangular box in S210, S220, and S230 shown in FIG. 2.Additionally, if the at least one moving object loiters in theparticular area for a preset period of time, for example, for 10 secondsor more, each time at which the at least one moving object loiters inthe particular area in a particular period of time may be stored. Anarea in which a fire has occurred, an area in which an accident hasoccurred, or an area in which monitoring is necessary may be set as aparticular area.

If a parking lot is set as a particular area, at least one moving objectthat appears continuously or stays near a stolen vehicle for aparticular period of time or more, for example for 30 minutes or moremay be detected.

In operation S620, an identifier is provided to each of the detected atleast one moving object. According to an exemplary embodiment, a path ofloitering of the at least one moving object may be tracked byidentifying each of the at least one moving object.

In operation S630, a path of each of the at least one moving object withan identifier is tracked. In operation S640, a representative stillimage of the at least one moving object in each time period is obtained.Then, in operation S650, the representative still image obtained in eachtime period with respect to each of the at least one moving object iscollected and displayed in an order of time.

According to an exemplary embodiment, respective representative stillimages, obtained in respective time periods with respect to a movingobject provided with an identifier, are collected and displayed in anorder of time. Thus, information about the identifier, information abouttime points at which the respective representative still images arecollected, or the like may be displayed in the form of an icon.

According to another exemplar embodiment, representative still imagesobtained in respective time periods with respect to a moving object withrespectfully identifiers are collected and displayed in the form of avideo summary. In this case, a path of the moving object may be furtherdisplayed.

As described above, according to the above exemplary embodiments, aloitering visualization apparatus may track a path of a moving object ofwhich loitering is detected in an order of time. Thus, even when aplurality of moving objects that are loitering in the scene aredetected, information about each moving object may be easily analyzed.

In addition, the above exemplary embodiments can also be implementedthrough computer-readable code/instructions in/on a medium, e.g., acomputer-readable medium, to control at least one processing element toimplement any above-described embodiment. The medium can correspond toany medium/media permitting the storage and/or transmission of thecomputer-readable code.

The computer-readable code can be recorded/transferred on a medium in avariety of ways, with examples of the medium including recording media,such as magnetic storage media (e.g., ROM, floppy disks, hard disks,etc.) and optical recording media (e.g., CD-ROMs, or DVDs), andtransmission media such as Internet transmission media. Thus, the mediummay be such a defined and measurable structure including or carrying asignal or information, such as a device carrying a bitstream accordingto one or more exemplary embodiments. The media may also be adistributed network, so that the computer-readable code isstored/transferred and executed in a distributed fashion. Furthermore,the processing element could include a processor or a computerprocessor, and processing elements may be distributed and/or included ina single device.

At least one of the components represented by a block as illustrated inFIG. 1 may be embodied as various numbers of hardware, software and/orfirmware structures that execute respective functions described above,according to an exemplary embodiment. For example, at least one of thesecomponents may use a direct circuit structure, such as a memory,processing, logic, a look-up table, etc. that may execute the respectivefunctions through controls of one or more microprocessors or othercontrol apparatuses. Also, at least one of these components may bespecifically embodied by a module, a program, or a part of code, whichcontains one or more executable instructions for performing specifiedlogic functions, and executed by one or more microprocessors or othercontrol apparatuses. Also, at least one of these components may furtherinclude a processor such as a central processing unit (CPU) thatperforms the respective functions, a microprocessor, or the like. Two ormore of these components may be combined into one single component,element or unit which performs all operations or functions of thecombined two or more components, elements of units. Also, at least partof functions of at least one of the components may be performed byanother of these components. Further, although a bus is not illustratedin the above block diagrams, communication between the components,elements or units may be performed through the bus. Functional aspectsof the above exemplary embodiments may be implemented in algorithms thatexecute on one or more processors. Furthermore, the componentsrepresented by a block or processing steps may employ any number ofrelated art techniques for electronics configuration, signal processingand/or control, data processing and the like.

It should be understood that the exemplary embodiments described thereinshould be considered in a descriptive sense only and not for purposes oflimitation. Descriptions of features or aspects within each embodimentshould typically be considered as available for other similar featuresor aspects in other embodiments.

While a plurality of exemplary embodiments have been described withreference to the drawings, it will be understood by those of ordinaryskill in the art that various changes in form and details may be madetherein without departing from the spirit and scope of the inventiveconcept as defined by the following claims.

What is claimed is:
 1. A method of visualizing loitering objects, the method comprising: detecting at least one object determined to have been in a selected area of an input image for a preset time period; obtaining representative still images of each of the detected at least one object in respective time periods during the preset time period; and performing at least one of displaying the representative still images in a time order and generating a video summary in which images of each of the detected at least one object, respectively included in the representative still images, are displayed together on a single image.
 2. The method of claim 1, further comprising assigning an identifier to each of the detected at least one object, wherein the identifier is displayed along with each of the detected at least one object on the representative still images or on the single image of the video summary.
 3. The method of claim 2, further comprising tracking a path of each of the detected at least one object to which the identifier is assigned, wherein the obtaining the representative still images is performed while tracking the path of each of the at least one object.
 4. The method of claim 1, wherein the displaying the representative still images in the time order is performed instead of the generating the video summary, and wherein the displaying the representative still images in the time order comprises displaying, in the time order, the representative still images along with respective information about time points when the representative still images are captured by a camera.
 5. The method of claim 4, further comprising assigning an identifier to each of the detected at least one object, wherein the identifier is displayed along with each of the detected at least one object on the representative still images.
 6. The method of claim 1, wherein the generating the video summary is performed instead of the displaying the representative still images in the time order, and wherein the generating the video summary comprises tracking a path of each of the detected at least one object in the time order, and indicating the path on the single image.
 7. The method of claim 6, further comprising assigning an identifier to each of the detected at least one object, and indicating the identifier on the single image.
 8. The method of claim 1, wherein the generating the video summary is performed instead of the displaying the representative still images in the time order, and wherein, in the generating the video summary, the images of each of the detected at least one object respectively included in the representative still images are displayed on the single image with indication of the time order.
 9. The method of claim 1, wherein each of the representative still images displays at least one of a front view, a side view, and a rear view of each of the at least one object.
 10. The method of claim 1, wherein the detected at least one object is at least one object which has appeared more than a predetermined number of times in the selected area for the preset time period.
 11. An apparatus for visualizing loitering objects, the apparatus comprising: an object detector configured to detect at least one object determined to have been in a selected area of an input image for a preset time period; an image obtainer configured to obtain representative still images of each of the detected at least one object in respective time periods during the preset time period; and a loitering visualizer configured to perform at least one of displaying the representative still images in a time order, or generating a video summary in which images of each of the detected at least one object, respectively included in the representative still images, are displayed together on a single image with indication of the time order.
 12. The apparatus of claim 11, further comprising an object identifier configured to assign an identifier to each of the detected at least one object, wherein the loitering visualizer is further configured to display the identifier along with each of the detected at least one object on the representative still images or on the single image of the video summary.
 13. The apparatus of claim 12, further comprising a path tracker configured to track a path of each of the detected at least one object to which the identifier is assigned, wherein the image obtainer is configured to obtain the representative still images while the path tracker tracks a path of each of the at least one object.
 14. The apparatus of claim 11, wherein the loitering visualizer is configured to display the representative still images in the time order instead of generating the video summary, and wherein the loitering visualizer is further configured to display, in the time order, the representative still images along with respective information about time points when the representative still images are captured by a camera.
 15. The apparatus of claim 14, further comprising an object identifier configured to assign an identifier to each of the detected at least one object, wherein the loitering visualizer is further configured to display the identifier along with each of the detected at least one object on the representative still images.
 16. The apparatus of claim 11, wherein the loitering visualizer is configured to generate the video summary instead of displaying the representative still images in the time order, and wherein the loitering visualizer is further configured to track a path of each of the detected at least one object in the time order, and indicate the path on the single image.
 17. The apparatus of claim 16, further comprising an object identifier configured to assign an identifier to each of the detected at least one object, wherein the loitering visualizer is further configured to indicate the identifier on the single image.
 18. The apparatus of claim 11, wherein the loitering visualizer is configured to generate the video summary instead of displaying the representative still images in the time order, and Wherein, in the generating the video summary, the loitering visualizer is configured to display indication of the time order along with the images of each of the detected at least one object on the single image.
 19. The apparatus of claim 11, wherein each of the representative still images displays at least one of a front view, a side view, and a rear view of each of the at least one object.
 20. The apparatus of claim 10, wherein the detected at least one object is at least one object which has appeared more than a predetermined number of times in the selected area of the input image for the preset time period. 